Evaluating Defect Detection Techniques for Software Requirements Inspections

Perspective-Based Reading (PBR) is a family of defect detection techniques which have been proposed to improve the effectiveness of software requirements inspections. PBR drives individual document reading by means of perspectivedependent procedural scenarios, which are different for each inspector in the team. Based on the former PBR experiments, we present a controlled experiment with more than one hundred undergraduate students who conducted software inspections as part of a university course. We tested an enhanced procedural version of PBR by comparing it ad hoc reading and checklist-based reading, and analyzing data both at the individual and team level. The experimental results do not support previous findings that PBR improves defect detection effectiveness with respect to nonsystematic reading techniques. We also investigated team composition issues by assessing the effects of combining different or identical perspectives at inspection meetings. The empirical test did not meet the assumption that different perspectives in an inspection team achieve a higher coverage of a document. Process conformance have been explicitly considered and used to check whether readers have departed from the assigned guidelines and then filter raw data. We discovered that only one fifth of PBR reviewers had actually followed their scenarios, and even many checklist reviewers had reverted back to an ad hoc reading style. From the debriefing questionnaires, we measured the reviewers’ self-assessment of the assigned reading techniques. The results revealed no relationship with the type of reading technique but the subjective evaluation was significantly related to trust in process conformance. This experiment provides evidence that process conformance issues play a critical role in the successful application of reading techniques and more generally, software process tools.